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Introduction 


The study, Maximizing Student Agency: Implementing and Measuring Student-Centered Learning 
Practices, aimed to identify the instructional practices that may be useful for the development of 
different aspects of student agency (i.e., self-efficacy, selfregulated learning, and persistence) and 
determine whether these instructional practices are equally helpful for different subgroups of 
students. In collaboration with four New Tech Network (NTN) high schools, the American Institutes 
for Research (AIR) used a mixed-methods approach consisting of teacher and student surveys and 


focus groups as well as facilitation of a networked improvement community (NIC) to address 


the research questions in Table 1. 


Table 1. Research Questions 


Focus Area Primary Research Question(s) 


Teacher practices designed to 
promote student agency 


What practices do teachers employ to provide 
feedback to students on their performance that 
assist with the development of student agency? 


How do teachers use data to inform their 
practices? 


Data Sources 


™ Teacher focus group data 
™ Menu of Teacher Practices 
™ Teacher survey data 

™ PDSA cycle data 

= NIC meeting data 


Contextual factors influencing What contextual factors do teachers view as 
the promotion of student facilitators of or challenges to implementing 
agency these practices? 


™ Teacher focus group data 


Lessons learned about 
surveying student agency 
over time 


How well do student survey questions measure 
student agency? 


Were the measurement properties of the agency 
scales consistent over time and across the 
student subgroups? 


™ Student survey data 


™ Student survey data 


Are there significant subgroup differences in 
measures of student agency? 


™ Student survey data 


How does student agency change during the 
school year? 


™ Teacher focus group data 
™ Student survey data 
= NIC meeting data 


Do changes in student agency during the school 
year differ between subgroups of students? 


™ Teacher focus group data 
™ Student survey data 
= NIC meeting data 


The technical appendix provides additional information about the study, including a description of 


the study sample, survey administration procedures, survey response rates, statistical analyses 
used to address research questions, and detailed study findings. A full accounting of results 
pertaining to the study’s research questions can be found in the final report, Maximizing Student 


Agency: Implementing and Measuring Student-Centered Learning Practices. 
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Study Sample 


NTN schools use project-based learning to empower and challenge students to learn and succeed, 
collaborate and communicate, and engage in the world around them. A critical component of 
their approach is student agency, or students’ capabilities to manage their own learning and be 
successful in school. Participating schools were recruited from an initial list of eight schools 
provided by our partners at NTN. Each of the eight schools had experience participating in research 
studies and worked closely with NTN on an existing continuous improvement initiative (the 
Assessment Improvement Community), and so NTN felt they were ready and able to participate 

in this study. From the initial list, four schools agreed to participate in the study. 


Data Sources 


AIR’s study team addressed the research questions (Table 1) using a mixed-methods approach. 
To address research questions associated with teacher practices designed to promote student 
agency and contextual factors influencing the promotion of student agency, the team conducted 
teacher focus groups in spring 2017 and spring 2018, and teacher surveys in fall 2017 and spring 
2018. To address research questions associated with surveying student agency over time, the 
team administered a student survey in fall 2017 and spring 2018. To address the research 
question associated with teachers’ use of data to inform instructional practices, the project 
team worked with a NIC comprised of teachers from the four study high schools. Data for the 
current study came from three primary sources: survey data, focus group data, and Plan-Do-Study- 
Act (PDSA) cycle data collected by the NIC. 


Student Survey Data 


In fall 2017 and spring 2018, the research team administered a student survey that included 
measures of student agency. In fall 2017, we collected survey data from 184 students attending 
the four participating schools.t A second survey was administered to 385 students (including 132 
of the students who took the fall survey) in spring 2018. Although the fall 2017 survey was limited 
to students in classes with NIC teachers, to facilitate analyses that look more closely at differences 
between subgroups of students, all students within participating schools were invited to participate 
in the spring 2018 survey. Overall, we analyzed data from 437 unique survey respondents. 


For each survey, students were asked to consider one of their classes when responding to survey 
questions. In the fall survey, students were asked to report on either (1) the NIC teacher in whose 
class they took the survey or, if they did not take the survey in a NIC teacher’s class, (2) the NIC 


1 This number excludes survey respondents who were removed from the data for a variety of reasons, including 
missing student name, students reporting that they did not take a class with a NIC teacher (in the fall), duplicate 
records where students reported twice about the same class, and missing data for all 54 items associated with 
the student agency scales. 
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teacher that they would see next.? In the spring, students who took the fall survey were asked to 
report on the class they reported on in the fall. Similar to the fall survey, students who had NIC 
teachers but who did not take the fall survey were asked to report on either (1) the NIC teacher 
in whose class they took the survey or, if they did not take the survey in a NIC teacher’s class, 
(2) the NIC teacher that they would see next. All students who responded to the spring survey but 
who did not have a teacher participating in the NIC were asked to report on the first academic 
class of the day. 


The distribution of survey respondents across schools and survey administrations is presented 
in Table 2. Response rates were calculated by counting any student who responded to the fall 
or spring survey as a survey respondent. Although survey response rates ranged from 21% to 48% 
across participating schools, response rates varied widely by grade level. Response rates by grade 
level are provided in Table 3. In general, response rates were highest for Grade 9 students (with 
response rates exceeding 70% in two participating schools) and lower for the higher grades 
(although one school had a 70% response rate among Grade 11 students). 


Table 2. Distribution of Consented Students and Student Survey Respondents Across Study Schools 


Survey 
Total Consented Spring Both Total Survey Response 

School Students Students Fall Only Only Surveys Respondents Rates 
School A 291 140 7 76 31 114 39.2% 
School B 298 144 17 65 27 109 36.6% 
School C 314 155 11 104 34 149 47.5% 
School D 310 66 17 8 40 65 21.0% 
Total 1,213 505 52 253 132 437 36.0% 


Table 3. Student Survey Response Rates Across Study Schools, by Grade Level 


School Grade 9 Grade 10 Grade 11 Grade 12° 
School A 71.9% 65.5% 14.3% 0.0% 
School B 71.1% 54.1% 13.2% 6.9% 
School C 68.9% 42.3% 53.6% 0.0% 
School D 13.8% 3.9% 69.7% 0.0% 
Total 52.1% 42.2% 36.1% 2.2% 


2 For each school, and in both the fall and spring surveys, NIC teachers were listed in a drop-down menu so that 
students could see the list of teachers who participated in the NIC. 

3 Data collection efforts were focused on students in Grades 9-11 as few NIC teachers taught students in 
Grade 12. All students were invited to complete the survey during the spring 2018 administration, yet we 
saw few Grade 12 responses. 
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Teacher Survey Data 


The research team administered surveys to 58 teachers in fall 2017 and 65 teachers in spring 
2018. The response rate increased from 75% in the fall to 82% in the spring, with response 
rates across schools ranging from 68% to 100% in the fall and from 77% to 90% in the spring 
(see Table 4). At both survey administrations, the response rates for NIC teachers (100% in the 
fall and 96% in the spring*) exceeded the response rate for non-NIC teachers (64% in the fall 
and 76% in the spring). 


Table 4. Distribution of Teacher Survey Respondents Across Study Schools 


Fall Survey Spring Survey 
Total Number of Response a0) e) | Number of Response 
School Teachers Respondents Rate Teachers Respondents Rate 
School A 9 9 100% 10 9 90% 
School B 19 13 68% 19 15 19% 
School C 23 18 18% 24 21 88% 
School D 26 18 69% 26 20 17% 
Total 77 58 75% 79 65 82% 


Focus Group Data 


In spring 2017, the research team conducted teacher focus groups® at each school. The AIR team 
conducted eight focus groups with a total of 40 teachers. Topics of discussion included the 
following five categories: 


Definitions of student agency, 

Goals for student agency in the classroom, 

Teacher practices and opportunities designed to promote student agency, 
Data currently collected on student agency, and 


Facilitators of and barriers to agency. 


Using the information provided by teachers during these focus groups, AIR developed the 
Menu of Teacher Practices, which guided the selection of practices the NIC tested. In spring 
2018, the research team conducted a second round of teacher focus groups to gather 
additional data about perceptions of student agency as well as NIC activities. Topics 

of discussion included the following: 


Learnings about student agency, 


Change ideas or practices implemented to develop student agency, 


4 There were two schools in which staffing changes (e.g., long-term substitute taking over a NIC classroom 
for a maternity leave, teachers leaving the NIC) impacted the response rate. 


5 The focus group protocol is available upon request. 


Maximizing Student Agency: Implementing and Measuring Student-Centered Learning Practices | TECHNICAL APPENDIX 


Measurement and data of change ideas, 

Student demonstrations of agency, and 

Facilitators of and barriers to agency. 
In spring 2018, AIR conducted student focus groups at each of the study schools. Participants 
were selected randomly from the list of consented students at each school. Alternates were 
identified, also randomly, in the event that selected students were not able to participate in the 


focus group. The number and grade level of students participating in each focus group is presented 
in Table 5. Students were asked to provide feedback on the following: 


Definitions of student agency, 
Opportunities they have been provided to employ agency, 


Instructional practices their teachers have used this year (aligned to the Menu 
of Teacher Practices), 


Skills those practices have helped develop, and 


= Ideas for improvement. 


Table 5. Student Focus Groups 


Grade Level 

Number of 
School Students Grade 9 Grade 10 Grade 11 Grade 12 
School A 10 X X 
School B 11 X X 
School C 10 X 
School D 17 X X X 
PDSA Cycle Data 


The research team aided 25 teachers participating in the NIC in choosing specific change ideas 

to implement within the classroom to increase student agency. The team guided NIC teachers in 
completing a PDSA cycle to develop change ideas, implement them, test their effectiveness, and 
refine them. As part of testing the effectiveness of their change ideas, teachers collected data 
throughout spring 2018. Examples of the types of data collected by the teachers include student 
responses to brief surveys (i.e., exit slips), students’ grades, workshop attendance, and work 
resubmission rates. Table 6 describes the measures used by NIC teachers. 


NIC teachers at two schools (School A and School B) addressed the same change ideas and 
collected the same data throughout their schools. Two teachers at School C collected data on 
their own change ideas. Three teachers at School D chose the same change idea but administered 
separate surveys, while one teacher at School D chose a different change idea. 
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Table 6. PDSA Cycle Data Measures 


School Change Idea Measure 


School A Administer a student self-reflection at the end of each week ~— Weekly self-reflection survey 
to measure mastery and resources used. 


School B Provide students with additional resources and feedback so _ Individual Assessment of Knowledge 
they can revise and improve their work. and Thinking (IAKT) survey 


Growth mindset survey 


School C Waive zero grade policy for students who seek out extra Writing agency survey 
help. Workshop attendance 
Provide students with personalized verbal feedback. Feedback survey 

School D Provide students more choice in support resources. Resources survey A 


Resources survey B 


Resources survey C 


Agency survey 


Provide one-on-one conferencing on IAKT assignments. Student writing grades 


Student and Teacher Survey Samples 


Table 7 presents the characteristics of the 437 students who responded to either the fall or spring 
survey. We also show sample characteristics separately for students who responded to the fall 
survey and students who responded to the spring survey. In general, the composition of the student 
sample was similar between survey administrations: the percentage of students in Grade 9 was 
larger than the percentage of students in higher grades; the percentage of students reporting about 
their social studies courses or interdisciplinary classes was larger than the percentage of students 
reporting on their mathematics, English language arts (ELA), or science courses; and about half of 
the sample was female. Approximately 40% of the sample was White, 20% was Black, between 
24% and 29% was Hispanic, and approximately 20% of students spoke a language other than 
English at home. Finally, across survey administrations, approximately 70% of students were 
classified as having a lower socioeconomic status (SES) because they had fewer than 100 books 
at home.® Because approximately 2% of students in the fall and 7% of students in the spring did 
not provide demographic information, examination of subgroup differences in measures of student 
agency are limited to 175 students in the fall and 354 students in the spring who provided all the 
demographic information. 


5 This measure of socioeconomic status is currently used in the Trends in International Mathematics and Science 
Study student survey (https://nces.ed.gov/timss/pdf/2015_8th_grade_Student_Questionnaire.pdf). Both student 
surveys also have measures of socioeconomic status based on the number of electronic devices students have in 
their homes. We decided to focus on the definition of socioeconomic status based on the number of books in 
students’ homes because observed differences in student agency were more consistent across measures of 
student agency using this definition. 
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Table 7. Composition of Student Survey Sample 


All Survey Respondents Respondents With Fall Respondents With 
(n=437) Data (n=184) Spring Data (n=385) 
Variable Number Percent Number Percent Number Percent 
School 
School A 114 26.1 38 20.7 107 27.8 
School B 109 24.9 44 23.9 92 23.9 
School C 65 14.9 57 31.0 48 12.5 
School D 149 34.1 45 24.5 138 35.8 
Grade Level 
Grade 9 187 42.8 94 51.1 167 43.4 
Grade 10 141 32.3 45 24.5 123 31.9 
Grade 11 104 23.8 40 21.7 95 24.7 
Grade 12 5 1.1 5 2.7 0 0.0 
Subject 
Math 52 11.9 33 17.9 43 11.2 
English 54 12.4 26 14.1 50 13.0 
Science 85 19.5 34 18.5 71 18.4 
Other/Interdisciplinary 125 28.6 44 23.9 116 30.1 
Social studies 121 27.7 47 25.5 105 27.3 
Gender 
Male 196 44.9 78 42.4 178 46.2 
Female 213 48.7 103 56.0 182 47.3 
Missing 28 6.4 3 1.6 25 6.5 
Race/Ethnicity 
White 181 41.4 76 41.3 162 42.1 
Black 88 20.1 39 21.2 77 20.0 
Hispanic 110 25.2 54 29.3 94 24.4 
Other 30 6.9 12 6.5 27 7.0 
Missing 28 6.4 3 1.6 25 6.5 
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All Survey Respondents Respondents With Fall Respondents With 


(n=437) Data (n=184) Spring Data (n=385) 
Variable Number Percent Number Percent Number Percent 
Language Status 
English spoken at home 321 73.5 142 17.2 286 74.3 
Another language spoken at home 84 19.2 39 21.2 70 18.2 
Missing 32 13 3 1.6 29 7.5 


Socioeconomic Status 


Fewer than 100 books at home 308 70.5 125 67.9 275 71.4 
At least 100 books at home 101 23.1 50 27.2 85 22.1 
Missing 28 6.4 9 4.9 25 6.5 


Table 8 presents the characteristics of the teachers who completed the fall 2017 and spring 2018 
teacher surveys. At each survey administration, more than half of survey respondents participated 
in the NIC, and nearly half of teachers provided instruction to multiple grade levels. Teachers were 
most likely to report teaching between 75 and 100 students in a day, and about half had taught at 
the high school level for 11 years or more. Finally, teachers were distributed across core academic 
subjects as well as other types of classes such as foreign language and electives. 


Table 8. Composition of Teacher Survey Sample 


Respondents With Fall Data Respondents With Spring Data 
(n=58) (n=65) 


Variable Number Percent Number Percent 


NIC Participation 


Participated in the NIC 33 56.9 35 53.9 
Did not participate in the NIC 25 43.1 30 46.2 
School 
School A 9 15.5 9 13.8 
School B 13 22.4 15 23.1 
School C 18 31.0 21 32.3 
School D 18 31.0 20 30.8 
Grade Level 
Grade 9 10 V7.2 12 18.5 
Grade 10 9 15.5 8 12.3 
Grade 11 9 15:5 8 12.3 
Grade 12 4 6.9 4 6.2 
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Respondents With Fall Data 


Respondents With Spring Data 


(n=58) (n=65) 
Variable Number Percent Number Percent 
Multiple grades 24 41.3 31 47.7 
Missing 2 3.0 2 Sul 
Subject 
Math 12 20.7 12 18.5 
English language arts 12 20.7 13 20.0 
Science 7 124 10 15.4 
Social studies 8 13.8 9 13.8 
Other/Interdisciplinary 12 20.7 11 16.9 
Multiple subjects 6 10.3 9 13.8 
Missing 1 1.7 1 1.5 
Number of Students Taught During a Typical Day 
Fewer than 50 4 6.9 7 10.8 
50-74 14 24.1 18 Qief 
75-100 24 41.4 30 46.2 
More than 100 14 24.1 9 13.9 
Missing 2 3.5 1 1.5 
Years of Teaching Experience in Grades 9-12 
1 year 3 5.2 1 1.5 
2-3 years 8 13.8 10 15.4 
4-5 years 11 19.0 10 15.4 
6-10 years 5 8.6 13 20.0 
11 or more years 30 51. 30 46.2 
Missing 1 1. 1 1.5 
Years of Teaching Experience in the School 
1 year 10 14.2 2 3.1 
2-3 years 10 17.2 22 33.9 
4-5 years 14 24.1 15 23.1 
6-10 years 13 22.4 18 27.7 
11 or more years 3 5.2 4 6.2 
Missing 8 13.8 4 6.2 
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Survey Measures 


A list of the student agency measures included in the student survey, along with references to the 
sources of the measures, is provided in Table 9. Each construct was measured with between four 
and nine survey items, and responses to survey items ranged from 1 (disagree) to 4 (strongly 
agree). For each survey construct, we calculated a scale score by averaging responses to relevant 
survey items. Averages and standard deviations for the student agency measures also are provided 
in Table 9.7 


Table 9. Student Agency Constructs, Sources, and Example Items 


Fall Spring 
Standard Standard 
Construct Source Example Item Average Deviation Average Deviation 
Self-efficacy Chen, Gully, & In general, | think that | can 3.07 0.60 3.03 0.61 
Eden, 2001 achieve goals that are 

important to me. 
Perseverance Duckworth & New ideas and projects 2.69 0.68 2.56 0.74 
of interest? Quinn, 2009 sometimes distract me from 

previous ones. 
Perseverance Duckworth & | finish whatever | begin. 2.88 0.66 2.84 0.67 
of effort Quinn, 2009 
Locus of Levenson, 1981 _| can pretty much determine 2.97 0.57 2.89 0.55 
control what will happen in my life. 
Mastery Midgley et al., An important reason why | do 2.67 0.72 2.60 0.75 
orientation 2000 my classwork is because | like 

to learn new things. 
Meta-cognitive — Pintrich & | ask myself questions to 2.66 0.67 2.63 0.64 
self-regulation DeGroot, 1990 make sure | understand the 

material | have been studying 

in this class. 
Self-regulated Consortium on | set aside time to do my 2.19 0.72 2.67 0.70 
learning Chicago School homework and study. 

Research, 2009 

Future Consortium on What | learn in class is 3.07 0.80 2.89 0.78 
orientation Chicago School necessary for success in the 


Research, 2009 


future. 


@ Items in the perseverance of interest construct were reverse-coded so that higher values indicate a higher level 


of perseverance. 


’ As described below, we found that the measurement properties of several agency measures improved after 
removing one or two survey items. The averages and standard deviations in Table 9 were calculated after 
removing these survey items. 
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In addition to the measures of student agency, student surveys included questions that capture 
key student background information, including gender, race/ethnicity (White, Black, Hispanic, or 
“other” racial/ethnic group), socioeconomic status, and English learner (EL) status (i.e., whether 
a language other than English is spoken at home). 


In the teacher survey, teachers were asked about the frequency with which they engaged in practices 
associated with increasing student agency? with most of their students. In addition, teachers were 
asked about how many students in their school (none, some, about half, most, or nearly all) have 
different types of learning opportunities. Finally, the teacher survey included survey items that 
allowed us to measure key aspects of the school setting (e.g., teachers’ commitment to the 
school, perceived program coherence, instructional improvement culture, self-efficacy for teaching). 


Methods 


Focus Group Analyses 


To identify the instructional practices used by teachers to develop agency and the facilitating 
factors and challenges in using those practices, the research team conducted teacher focus 
groups. Focus groups were recorded, transcribed, and coded using a coding structure aligned to 
the research questions. The research team analyzed the focus group data to identify both themes 
and outliers within the responses. The research team used data from the spring 2017 focus groups 
to catalog and categorize the instructional practices identified by participants, capture themes 
related to the key elements of those practices and shared relevant quotes that illustrated how 
teachers used the practice and develop the Menu of Teacher Practices. The practices listed in 
the Menu of Teacher Practices were included in a teacher survey that was administered in fall 
2017 and spring 2018, allowing the research team to calculate the percentage of teachers 
who reported using specific practices with most of their students more than three times a week. 
Finally, to identify perceptions of the facilitators and challenges in implementing those practices, 
the team used data from both the spring 2017 and spring 2018 focus groups. Responses were 
grouped by theme and sorted based on frequency of response to identify the primary facilitators 
and challenges. 


Examination of Measurement Properties 


The first stage of survey data analysis involved the assessment of the survey measures of student 
agency. The research team calculated Cronbach's alpha (a measure of internal consistency) and 
conducted confirmatory factor analysis (CFA) to examine the measurement properties of the 
student agency constructs. CFAs tested whether previously validated survey measures did a good 
job of measuring intended constructs within our survey sample by calculating model fit statistics, 
such as the root mean square error of approximation (RMSEA) and the comparative fit index (CFI). 
For these analyses, we used the following cut-offs to indicate a good fit to the data: alpha > 0.70, 
CFI > 0.95, RMSEA < 0.10.29 


8 The practices included in the teacher survey reflected the instructional practices identified during the spring 2017 
focus groups and outlined within the Menu of Teacher Practices. 


° Small sample sizes reduce the likelihood of achieving ideal model fit. 
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Tests of Measurement Invariance 


To measure changes in student agency over time, it was necessary to determine whether the 
measurement properties of the agency scales were consistent over time. In addition, to compare 
levels of student agency across student subgroups, it was necessary to determine whether the 
measurement properties of the agency scales were consistent across student subgroups. We 
examined measurement invariance of student agency measures by the following student 
characteristics: grade level, gender, race/ethnicity, English learner (EL) status, socioeconomic 
status (SES), and subject area. We examined both metric invariance (i.e., regression weights 
were not equivalent across groups) and scalar invariance (i.e., at a given level of the student 
agency measure, the intercept of individual survey items was not equivalent across groups). 

In addition to examining the significance of chi-square model fit statistics tests, we looked at 
meaningful changes in the CFl and RMSEA values between models that did and did not constrain 
the regression weights and item intercepts to be equivalent across groups. Measures were 
classified as having invariance issues if (1) the difference in chi-square model fit tests between 
constrained models (where regression weights and intercepts were constrained to be equal across 
groups) and unconstrained models (where regression weights and/or intercepts were allowed to 
vary across groups) were significant at the .1 confidence level AND (2) values of RMSEA differed 
by .015 or more or values of CFI differed by 0.01 or more between constrained and unstrained 
models. These tests allowed us to determine whether student agency measures worked equally 
well for different groups of students. 


Comparing Levels of Student Agency Across Student Subgroups 


To examine subgroup differences in levels of student agency in the fall and spring, we estimated 
two-level ordinary least squares (OLS) regression models where students are nested within 
schools. We examined differences in levels of student agency based on the following student 
characteristics: grade level, gender, race/ethnicity, EL status, SES level, and subject area. All 
student subgroup indicators were included in a single regression model so that, for instance, 
racial/ethnic gaps in student agency measures accounted for racial/ethnic differences in SES 
level or EL status. 


Examining Change in Student Agency During the School Year 


Similar two-level OLS regression models were estimated to examine changes in student agency 
over time as well as subgroup differences in changes in student agency measures. For these 
statistical models, the outcome of interest was change in student agency, which was measured 
as the spring value of the student agency measure minus the fall value of the student agency 
measure. To estimate average change over time within the student sample, all subgroup indicators 
were centered within schools. To estimate subgroup differences in change over time, models 
included uncentered subgroup indicators, and all subgroup indicators were included in the same 
statistical model. 
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Examination of PDSA Data 


To identify how teachers are using data to inform their practices, the research team examined notes 
from NIC meetings and PDSA cycle data provided to the team by the participating teachers. We used 
these sources of information to summarize the data collection methods used by the NIC teachers, 
and how teachers used these data to test the effectiveness of their instructional approaches. 


Detailed Study Findings 


Analysis of the Quality of Survey Measures 


To examine change in survey measures over time and compare levels of student agency across 
student subgroups, it was necessary to examine the measurement properties of the student 
agency scales. Results of these analyses indicated that, for measures of future orientation, locus 
of control, and metacognitive self-regulation, measurement of the constructs improved after removing 
one or two items (see Table 10). The standardized regression weights associated with the original 
and revised measures are presented in Tables 11, 12, and 13 for future orientation, locus of 
control, and metacognitive self-regulation, respectively. 


Table 10. Model Fit Statistics for Confirmatory Factor Analysis Models Estimating Measures of 
Student Agency 


Student Agency Measure CFI RMSEA Cronbach’s Alpha 
Self-efficacy 0.953 0.109 0.90 
Perseverance (original—both interest and effort) 0.953 0.076 0.70 
Perseverance of interest 0.981 0.102 0.71 
Perseverance of effort 0.996 0.050 0.75 
Locus of contro! (original) 0.928 0.086 0.75 
Locus of control (excluding item 3) 0.966 0.070 0.75 
Mastery orientation 0.968 0.111 0.88 
Metacognitive self-regulation (original) 0.874 0.111 0.85 
Metacognitive self-regulation (excluding items 1 and 7) 0.958 0.078 0.89 
Self-regulated learning 0.976 0.065 0.91 
Future orientation (original) 0.970 0.120 0.87 
Future orientation (excluding item 1) 0.999 0.033 0.89 
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Table 11. Standardized Regression Weights for Future Orientation Survey Items, Original Measure, and 
After Removing Problematic Item 


After Removing 


To what extent do you agree or disagree with the following statements Original Problematic 
about you? Measure Survey Item 
1. Grades in high school matter for success in college. 0.57 Removed 
2. My classes give me useful preparation for what | plan to do in life. 0.80 0.81 

3. High school teaches me valuable skills. 0.82 0.82 

4. Working hard in high school matters for success in the workforce. 0.76 0.74 

5. What! learn in class is necessary for success in the future. 0.86 0.87 


Table 12. Standardized Regression Weights for Locus of Control Survey Items, Original Measure, and 
After Removing Problematic Item 


After Removing 


To what extent do you agree or disagree with the following statements Original Problematic 
about you? Measure Survey Item 
1. Whether or not | get to be a leader depends mostly on my ability. 0.47 0.46 

2. When | make plans, | am almost certain to make them work. 0.61 0.61 

3. How many friends | have depends on how nice a person | am. 0.33 Removed 
4. |can pretty much determine what will happen in my life. 0.49 0.47 

5. | am usually able to protect my personal interests. 0.72 0.72 

6. When | get what | want, it’s usually because | worked hard for it. 0.70 0.72 

7. My life is determined by my own actions. 0.62 0.62 


Table 13. Standardized Regression Weights for Metacognitive Self-Regulation Survey Items, Original 
Measure, and After Removing Problematic Items 


After Removing 


To what extent do you agree or disagree with the following statements Problematic 

about you? Original Measure Survey Item 

1. During class time | often miss important points because I'm thinking 0.05 Removed 
of other things. 

2. When become confused about something I'm reading for this class, 0.66 0.66 


| go back and try to figure it out. 


3. If class materials are difficult to understand, | change the way | read 0.72 0.71 
the material. 
4. Before | study new class material thoroughly, | often skim it to see how 0.66 0.66 


it is organized. 


5. | ask myself questions to make sure | understand the material | have 0.75 0.75 
been studying in this class. 
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After Removing 


To what extent do you agree or disagree with the following statements Problematic 

about you? Original Measure Survey Item 

6. | try to change the way | study in order to fit the class requirements and 0.74 0.74 
instructor's teaching style. 

7. | often find that | have been reading for class but don't know what it was 0.26 Removed 
all about. 

8. | try to think through a topic and decide what | am supposed to learn 0.60 0.59 
from it rather than just reading it over when studying. 

9. When studying for this class, | try to determine which concepts | don't 0.74 0.74 
understand well. 

10. When | study for this class, | set goals for myself in order to direct my 0.75 0.75 
activities. 

11. If | get confused taking notes in class, | make sure | figure it out afterward. 0.73 0.73 


Measurement Invariance 


Across the student agency measures, we did not observe a consistent pattern of differing 
measurement properties across multiple subgroups of students. However, we found a few 
instances where the measurement properties of student agency measures were not equal 
across different student subgroups. Results of these analyses are presented below. 


Differences Between Fall and Spring Survey Administrations 


Results confirmed that the ways in which survey responses related to one another did not change 
over time, allowing us to examine change over time, with one exception: We did not observe scalar 
invariance for the measure of future orientation. An examination of estimated intercepts indicates 
that, for a given level of future orientation, responses to individual survey items were generally 
higher in the fall than they were in the spring (Table 14). 


Table 14. Results of Measurement Invariance Tests, Fall Versus Spring Survey Administration 


Metric Invariance Scalar Invariance Notes on CFI and RMSEA 


(p-value) (p-value) if p-value Is Less Than .1 Invariant? 
Self-efficacy 0.729 0.115 N/A Yes 
Perseverance of interest 0.808 0.084 CFI improved by .007; Yes 
RMSEA improved by .009 

Perseverance of effort 0.987 0.410 N/A Yes 
Locus of control 0.408 0.216 N/A Yes 
(excluding item 3) 

Mastery orientation 0.802 0.750 N/A Yes 
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Metric Invariance Scalar Invariance Notes on CFI and RMSEA 


(p-value) (p-value) if p-value Is Less Than .1 Invariant? 


Metacognitive self- 0.930 0.779 N/A Yes 
regulation (excluding 
items 1 and 7) 


Self-regulated learning 0.740 0.491 N/A Yes 
Future orientation 0.301 0.013 CFI improved by .007; No 
(excluding item 1) RMSEA improved by .019 


Note. Numbers in red indicate a significant improvement in model fit by allowing measurement properties to differ between the fall and 
spring survey administrations. 


Differences by Subject Area 


Measures of locus of control and self-regulated learning did not work equally well across academic 
subjects (Tables 15-17). The second survey item for locus of control (“When | make plans, | am 
almost certain to make them work”) did not work as well in English language arts (ELA) or social 
studies classes as it did in other classes, while the third survey item (“I can pretty much 
determine what will happen in my life”) did not work as well in mathematics or social studies as it 
did in other classes. For the measure of self-regulated learning, the third and fourth items (“If | 
need to study, | don’t go out with my friends” and “I always study for tests”) did not work as well in 
ELA or interdisciplinary classes as they did for other classes. 


Table 15. Results of Measurement Invariance Tests, by Subject Area 


iY Cexaa tom Uielitelilers) Tere] elm eliteliles) Notes on CFI and RMSEA 


(p-value) (p-value) if p-value Is Less Than .1 Invariant? 
Self-efficacy 0.971 0.283 N/A Yes 
Perseverance of interest 0.119 0.523 N/A Yes 
Perseverance of effort 0.131 0.175 N/A Yes 
Locus of control 0.003 0.011 CFI improved by .040 and No 
(excluding item 3) .033; RMSEA improved by 
.004 and .000 

Mastery orientation 0.865 0.506 N/A Yes 
Metacognitive self- 0.845 0.136 N/A Yes 


regulation (excluding 
items 1 and 7) 


Self-regulated learning 0.018 0.176 CFI improved by .010; No 
RMSEA improved by .001 
Future orientation 0.886 0.542 N/A Yes 


(excluding item 1) 


Note. Numbers in red indicate a significant improvement in model fit by allowing measurement properties to differ by subject area. 
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Table 16. Standardized Regression Weights for Locus of Control Survey Items, by Subject Area 


English 

To what extent do you agree or disagree Language Social 

with the following statements about you? Mathematics Arts Science Studies  Interdisciplinary 

1. Whether or not | get to be a leader 0.62 0.65 0.66 0.58 0.58 
depends mostly on my ability. 

2. When! make plans, | am almost 0.77 0.22 0.59 0.30 0.52 
certain to make them work. 

3. | can pretty much determine what will 0.08 0.56 0.63 0.35 0.60 
happen in my life. 

4. |am usually able to protect my 0.84 0.45 0.82 0.75 0.62 
personal interests. 

5. When | get what | want, it’s usually 0.74 0.76 0.68 0.78 0.65 
because | worked hard for it. 

6. My life is determined by my own 0.63 0.58 0.64 0.67 0.53 
actions. 


Table 17. Standardized Regression Weights for Self-Regulated Learning Survey Items, by Subject Area 


English 

To what extent do you agree or disagree Language Social 

with the following statements about you? Mathematics Arts Science Studies  Interdisciplinary 

1. | set aside time to do my homework 0.75 0.68 0.87 0.76 0.78 
and study. 

2. | try to do well on my schoolwork even 0.62 0.76 0.75 0.73 0.62 
when it isn’t interesting to me. 

3. If | need to study, | don’t go out with my 0.74 0.49 0.72 0.68 0.56 
friends. 

4. | always study for tests. 0.75 0.51 0.79 0.78 0.53 

5. | keep track of my long-term 0.76 0.74 0.83 0.79 0.72 
assignments so | know when to turn 
them in. 

6. | manage my time well enough to get 0.73 0.90 0.86 0.7 0.83 
all my work done. 

7. |can keep my schoolwork and personal 0.78 0.89 0.75 0.73 0.78 
life organized. 

8. | set goals for my performance in 0.84 0.71 0.81 0.75 0.84 
Classes. 

9. | have a system for organizing my 0.72 0.77 0.78 0.74 0.72 
schoolwork. 
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Differences by Grade Level 


Measures of student agency were invariant across grade levels, with one exception: The measure 
of locus of control did not work similarly well across grade levels (see Table 18). As shown in Table 
19, the standardized regression weights differed by grade level such that some items were more 
strongly related for Grade 9 students (e.g., “Whether or not | get to be a leader depends mostly on 
my ability”) and some items were more strongly related for Grade 10 and Grade 11 students (e.g., 
“When | get what | want, it’s usually because | worked hard for it”). In addition, the item “When | 
make plans, | am almost certain to make them work” loaded particularly weakly to the construct of 
locus of control for Grade 10 students only. These findings suggest that survey items did not work 
similarly well across grade levels. 


Table 18. Results of Measurement Invariance Tests, by Grade Level 


Celta Com i\eclatelirers) Scalar Invariance Notes on CFI and RMSEA 


(p-value) (p-value) if p-value Is Less Than .1 Invariant? 
Self-efficacy 0.959 0.351 N/A Yes 
Perseverance of interest 0.789 0.517 N/A Yes 
Perseverance of effort 0.116 0.125 N/A Yes 
Locus of control 0.063 0.512 CFI improved by .013; No 
(excluding item 3) RMSEA did not improve 
Mastery orientation 0.051 0.067 CFI improved by Yes 


.006 and .005; RMSEA 
improved by .006 and .007 


Metacognitive self- 0.860 0.277 N/A Yes 
regulation (excluding 
items 1 and 7) 


Self-regulated learning 0.320 0.256 N/A Yes 
Future orientation 0.935 0.046 CFI improved by .009; Yes 
(excluding item 1) RMSEA improved by .004 


Note. Numbers in red indicate a significant improvement in model fit by allowing measurement properties to differ by grade level. 


Table 19. Standardized Regression Weights for Locus of Control Survey Items, by Grade Level 


To what extent do you agree or disagree with the following statements 


about you? eTeTe(smy) Grade 10 Grade 11 
1. Whether or not | get to be a leader depends mostly on my ability. 0.69 0.54 0.57 
2. When! make plans, | am almost certain to make them work. 0.56 0.29 0.54 
3. | can pretty much determine what will happen in my life. 0.46 0.44 0.55 
4. 1am usually able to protect my personal interests. 0.70 0.71 0.76 
5. When | get what | want, it’s usually because | worked hard for it. 0.66 0.80 0.72 
6. My life is determined by my own actions. 0.51 0.69 0.71 
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Differences by Race/Ethnicity 


The measure of perseverance of interest did not work equally well for White and non-White 
students (see Table 20). Further examination revealed that regression weights were stronger for 
non-White students for three of the four survey items, while regression weights were more similar 
in magnitude for the survey item “I have difficulty maintaining my focus on projects that take more 
than a few months to complete” (see Table 21). In addition, mastery orientation did not achieve 
scalar invariance. An examination of estimated intercepts revealed that, given the same level of 
mastery orientation, the intercept for the first item (“I like classwork that I’ll learn from even if | 
make a lot of mistakes”) was higher for White students, and the intercept for the fourth and fifth 
items (“An important reason why | do my classwork is because | want to get better at it” and “An 
important reason | do my classwork is because | enjoy it”) was higher for non-White students. 


Table 20. Results of Measurement Invariance Tests, by Race/Ethnicity 


Metric Invariance Scalar Invariance Notes on CFI and RMSEA 


(p-value) (p-value) if p-value Is Less Than .1 Invariant? 

Self-efficacy 0.551 0.122 N/A Yes 

Perseverance of interest 0.039 0.431 CFI improved by .011; No 
RMSEA improved by .004 

Perseverance of effort 0.639 0.497 N/A Yes 

Locus of control 0.137 0.678 N/A Yes 

(excluding item 3) 

Mastery orientation 0.560 0.000 CFI improved by .015; No 
RMSEA improved by .007 

Metacognitive self- 0.496 0.025 CFI improved by .006; Yes 

regulation (excluding RMSEA improved by .002 

items 1 and 7) 

Self-regulated learning 0.378 0.079 CFI improved by .003; Yes 


RMSEA improved by .001 


Future orientation 0.641 0.369 N/A Yes 
(excluding item 1) 


Note. Numbers in red indicate a significant improvement in model fit by allowing measurement properties to differ by race/ethnicity. 
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Table 21. Standardized Regression Weights for Perseverance of Interest Survey Items, for White and 


Non-White Students 
White Non-White 
To what extent do you agree or disagree with the following statements about you? Students Students 
1. | often set a goal but later choose to pursue a different one. 0.56 0.67 
2. | have been obsessed with a certain idea or project for a short time but later 0.57 0.77 


lost interest. 


3. | have difficulty maintaining my focus on projects that take more than a few 0.84 0.77 
months to complete. 


4. New ideas and projects sometimes distract me from previous ones. 0.61 0.77 


Differences by Gender 


Several measures of student agency did not achieve scalar invariance by gender: perseverance 
of interest, locus of control, and future orientation. The remaining measures of student agency 
achieved measurement invariance by gender (see Table 22). Specifically, at a given level of 
perseverance of interest or locus of control, intercepts were generally higher for male students 
than for female students. In contrast, at a given level of future orientation, intercepts were 
generally higher for female students than for male students. 


Table 22. Results of Measurement Invariance Tests, by Gender 


Metric Invariance Scalar Invariance Notes on CFl and RMSEA 


(p-value) (p-value) if p-value Is Less Than .1 Invariant? 

Self-efficacy 0.949 0.716 N/A Yes 

Perseverance of interest 0.228 0.016 CFI improved by .018; No 
RMSEA improved by .004 

Perseverance of effort 0.428 0.225 N/A Yes 

Locus of control 0.263 0.029 CFI improved by .015; No 

(excluding item 3) RMSEA improved by .002 

Mastery orientation 0.182 0.029 CFI improved by .006; Yes 
RMSEA improved by .002 

Metacognitive self- 0.025 0.168 CFI improved by .005; Yes 

regulation (excluding RMSEA improved by .001 

items 1 and 7) 

Self-regulated learning 0.278 0.095 CFI improved by .003; Yes 
RMSEA improved by .003 

Future orientation 0.629 0.028 CFI improved by .008; No 

(excluding item 1) RMSEA improved by .018 


Note. Numbers in red indicate a significant improvement in model fit by allowing measurement properties to differ by gender. 
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Differences by SES 


Several measures of student agency did not achieve scalar invariance by SES level: perseverance 
of effort, locus of control, and metacognitive self-regulation. The remaining measures of student 
agency achieved measurement invariance by SES level (See Table 23). Specifically, at a given level 
of perseverance of effort or locus of control or metacognitive self-regulation, intercepts were 
generally higher for higher SES students than for lower SES students. 


Table 23. Results of Measurement Invariance Tests, by SES Level 


iY Cex datom Unieclirelilers) Scalar Invariance Notes on CFl and RMSEA 


(p-value) (p-value) if p-value Is Less Than .1 Invariant? 

Self-efficacy 0.568 0.010 CFI improved by 0.006; Yes 
RMSEA improved by 0.001 

Perseverance of interest 0.290 0.939 N/A Yes 

Perseverance of effort 0.729 0.093 CFI improved by .004; No 
RMSEA improved by .021 

Locus of control 0.428 0.003 CFI improved by .025; No 

(excluding item 3) RMSEA improved by .006 

Mastery orientation 0.300 0.004 CFI improves by .009; Yes 
RMSEA improved by .001 

Metacognitive self- 0.259 0.003 CFI improved by .01; No 

regulation (excluding RMSEA improved by .001 

items 1 and 7) 

Self-regulated learning 0.945 0.029 CFI improved by .005; Yes 
RMSEA improved by .001 

Future orientation 0.351 0.143 N/A Yes 


(excluding item 1) 


Note. Numbers in red indicate a significant improvement in model fit by allowing measurement properties to differ by SES level. 


Differences by EL Status 
All measures of student agency were invariant by EL status (see Table 24). 


Table 24. Results of Measurement Invariance Tests, by EL Status 


WY Keata tom (i eclitelirers) Scalar Invariance Notes on CFI and RMSEA if 


(p-value) (p-value) p-value Is Less Than .1 Invariant? 
Self-efficacy 0.001 0.340 CFI improved by .009; Yes 
RMSEA improved by .002 
Perseverance of interest 0.131 0.339 N/A Yes 
Perseverance of effort 0.312 0.497 N/A Yes 
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Keita tom (i\eclatelilers) Scalar Invariance Notes on CFI and RMSEA if 


(p-value) (p-value) p-value Is Less Than .1 Invariant? 
Locus of control 0.596 0.958 N/A Yes 
(excluding item 3) 
Mastery orientation 0.552 0.074 CFI improved by .004; Yes 
RMSEA improved by .003 
Metacognitive self- 0.610 0.343 N/A Yes 


regulation (excluding 
items 1 and 7) 


Self-regulated learning 0.284 0.455 N/A Yes 


Future orientation 0.161 0.550 N/A Yes 
(excluding item 1) 


Note. Numbers in red indicate a significant improvement in model fit by allowing measurement properties to differ by EL status. 


Regression Results: Subgroup Differences in Levels of Student Agency 


Findings from the fall and spring reveal somewhat different patterns of subgroup differences (see 
Tables 25-32). 


Table 25. Subgroup Differences in Self-Efficacy, in Fall and Spring 


Fall Survey (n=175) Spring Survey (n=354) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female 0.045 0.092 0.624 0.092 0.064 0.149 


Race/Ethnicity (White reference) 


Black -0.005 0.128 0.971 0.146 0.089 0.101 
Hispanic -0.186 0.136 0.169 0.048 0.093 0.611 
Other -0.297 0.192 0.123 -0.022 0.125 0.863 


English Learner (EL) Status (non-EL reference) 
EL 0.103 0.123 0.403 -0.179 0.091 0.049 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home 0.104 0.114 0.359 0.188 0.080 0.019 


Grade Level (Grade 9 reference) 


Grade 10 0.021 0.147 0.884 -0.041 0.085 0.629 


Grade 11 0.121 0.120 0.311 0.092 0.084 0.272 
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Fall Survey (n=175) Spring Survey (n=354) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Subject Area (social studies reference) 


Mathematics -0.040 0.140 0.774 -0.119 0.119 0.317 
English language arts 0.201 0.167 0.228 0.034 0.118 0.770 
Science 0.179 0.149 0.230 0.157 0.105 0.135 
Interdisciplinary -0.014 0.155 0.928 -0.002 0.100 0.983 
Constant 2.964 0.149 0.000 2.912 0.106 0.000 


Table 26. Subgroup Differences in Perseverance of Interest, in Fall and Spring 


Fall Survey (n=175) Spring Survey (n=353) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female 0.226 0.099 0.022 0.081 0.080 0.311 
Race/Ethnicity (White reference) 

Black 0.157 0.138 0.255 -0.046 0.112 0.682 

Hispanic -0.054 0.147 0.714 0.012 0.118 0.917 

Other 0.194 0.208 0.351 -0.080 0.157 0.610 


English Learner (EL) Status (non-EL reference) 


EL 0.010 0.134 0.938 -0.040 0.114 0.727 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home -0.073 0.123 0.552 0.114 0.102 0.265 


Grade Level (Grade 9 reference) 
Grade 10 -0.147 0.159 0.354 -0.084 0.107 0.432 
Grade 11 -0.131 0.130 0.312 0.119 0.109 0.275 


Subject Area (social studies reference) 


Mathematics 0.124 0.151 0.414 0.023 0.154 0.882 
English language arts 0.109 0.181 0.545 -0.007 0.151 0.961 
Science 0.359 0.161 0.026 -0.158 0.137 0.249 
Interdisciplinary 0.010 0.167 0.954 0.018 0.125 0.883 
Constant 2.510 0.161 0.000 2.513 0.144 0.000 
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Table 27. Subgroup Differences in Perseverance of Effort, in Fall and Spring 


Fall Survey (n=175) Spring Survey (n=353) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female 0.091 0.100 0.364 -0.015 0.069 0.825 


Race/Ethnicity (White reference) 


Black 0.072 0.139 0.603 0.091 0.096 0.345 
Hispanic -0.123 0.148 0.403 0.017 0.101 0.867 
Other -0.180 0.209 0.389 -0.078 0.136 0.565 


English Learner (EL) Status (non-EL reference) 
EL -0.102 0.134 0.450 -0.147 0.099 0.137 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home 0.142 0.124 0.251 0.186 0.087 0.033 


Grade Level (Grade 9 reference) 
Grade 10 0.035 0.160 0.828 -0.030 0.093 0.750 
Grade 11 0.076 0.130 0.561 0.158 0.091 0.084 


Subject Area (social studies reference) 


Mathematics 0.150 0.152 0.327 -0.113 0.129 0.383 
English language arts 0.394 0.182 0.030 0.081 0.128 0.527 
Science 0.342 0.163 0.036 0.275 0.114 0.016 
Interdisciplinary 0.186 0.168 0.269 0.021 0.108 0.846 
Constant 2.626 0.162 0.000 2.746 0.116 0.000 


Table 28. Subgroup Differences in Locus of Control, in Fall and Spring 


Fall Survey (n=175) Spring Survey (n=354) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female -0.050 0.086 0.560 -0.055 0.058 0.350 


Race/Ethnicity (White reference) 


Black 0.150 0.120 0.209 0.003 0.081 0.971 
Hispanic 0.005 0.127 0.970 0.063 0.085 0.458 
Other -0.238 0.181 0.187 -0.054 0.114 0.635 
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Fall Survey (n=175) Spring Survey (n=354) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


English Learner (EL) Status (non-EL reference) 


EL -0.039 0.116 0.739 -0.164 0.083 0.049 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home 0.094 0.107 0.378 0.086 0.073 0.240 


Grade Level (Grade 9 reference) 
Grade 10 -0.017 0.138 0.902 -0.009 0.078 0.903 
Grade 11 -0.185 0.113 0.100 0.048 0.077 0.537 


Subject Area (social studies reference) 


Mathematics 0.252 0.132 0.056 0.134 0.109 0.218 
English language arts 0.449 0.157 0.004 0.199 0.109 0.067 
Science 0.271 0.140 0.054 0.216 0.096 0.024 
Interdisciplinary 0.110 0.145 0.448 0.103 0.091 0.257 
Constant 2.810 0.140 0.000 2.801 0.097 0.000 


Table 29. Subgroup Differences in Mastery Orientation, in Fall and Spring 


Fall Survey (n=175) Spring Survey (n=354) 


SITIO Ce} Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female -0.076 0.106 0.473 0.022 0.079 0.781 
Race/Ethnicity (White reference) 

Black 0.280 0.148 0.059 0.081 0.111 0.465 

Hispanic 0.180 0.157 0.252 0.198 0.117 0.091 

Other -0.019 0.223 0.931 -0.006 0.156 0.972 


English Learner (EL) Status (non-EL reference) 


EL -0.017 0.143 0.906 -0.311 0.113 0.006 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home 0.283 0.132 0.032 0.208 0.101 0.038 


Grade Level (Grade 9 reference) 
Grade 10 0.159 0.170 0.351 -0.095 0.106 0.370 
Grade 11 0.198 0.139 0.155 0.078 0.107 0.467 
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Fall Survey (n=175) Spring Survey (n=354) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Subject Area (social studies reference) 


Mathematics -0.020 0.162 0.903 -0.140 0.152 0.355 
English language arts 0.468 0.194 0.016 0.046 0.150 0.760 
Science 0.197 0.173 0.256 0.085 0.134 0.524 
Interdisciplinary 0.082 0.179 0.646 -0.072 0.124 0.559 
Constant 2.318 0.172 0.000 2.579 0.138 0.000 


Table 30. Subgroup Differences in Metacognitive Self-Regulation, in Fall and Spring 


Fall Survey (n=175) Spring Survey (n=352) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female 0.159 0.098 0.103 0.096 0.066 0.149 
Race/Ethnicity (White reference) 

Black 0.290 0.136 0.032 0.087 0.092 0.344 

Hispanic 0.090 0.144 0.534 0.115 0.097 0.235 

Other -0.082 0.205 0.690 0.052 0.130 0.687 


English Learner (EL) Status (non-EL reference) 
EL 0.046 0.131 0.724 -0.285 0.095 0.003 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home 0.308 0.121 0.011 0.128 0.083 0.123 


Grade Level (Grade 9 reference) 
Grade 10 0.131 0.156 0.404 0.011 0.088 0.905 
Grade 11 0.185 0.127 0.146 0.099 0.087 0.258 


Subject Area (social studies reference) 


Mathematics 0.111 0.149 0.456 -0.178 0.123 0.149 
English language arts 0.507 0.178 0.004 0.106 0.122 0.387 
Science 0.272 0.159 0.086 0.199 0.109 0.068 
Interdisciplinary 0.008 0.165 0.962 0.025 0.103 0.805 
Constant 2.165 0.158 0.000 2.499 0.110 0.000 
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Table 31. Subgroup Differences in Self-Regulated Learning, in Fall and Spring 


Fall Survey (n=175) Spring Survey (n=353) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female 0.111 0.107 0.300 0.102 0.073 0.162 


Race/Ethnicity (White reference) 


Black 0.221 0.149 0.138 0.191 0.101 0.058 
Hispanic 0.172 0.158 0.275 0.153 0.106 0.149 
Other -0.074 0.224 0.741 0.092 0.142 0.516 


English Learner (EL) Status (non-EL reference) 


EL -0.030 0.144 0.835 -0.207 0.103 0.046 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home 0.372 0.133 0.005 0.286 0.091 0.002 


Grade Level (Grade 9 reference) 
Grade 10 0.137 0.171 0.425 -0.113 0.097 0.244 
Grade 11 0.162 0.139 0.244 0.113 0.096 0.237 


Subject Area (social studies reference) 


Mathematics 0.082 0.163 0.615 -0.229 0.135 0.090 
English language arts 0.372 0.194 0.056 -0.051 0.136 0.709 
Science 0.357 0.174 0.040 0.158 0.119 0.184 
Interdisciplinary 0.109 0.180 0.546 -0.031 0.113 0.786 
Constant 2.303 0.173 0.000 2.529 0.121 0.000 


Table 32. Subgroup Differences in Future Orientation, in Fall and Spring 


Fall Survey (n=175) Spring Survey (n=353) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female 0.123 0.117 0.294 0.173 0.080 0.030 


Race/Ethnicity (White reference) 


Black 0.267 0.163 0.102 0.123 0.111 0.268 
Hispanic 0.290 0.173 0.094 0.191 0.117 0.101 
Other 0.140 0.246 0.569 -0.009 0.157 0.956 
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Fall Survey (n=175) Spring Survey (n=353) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


English Learner (EL) Status (non-EL reference) 


EL -0.033 0.158 0.832 -0.330 0.114 0.004 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home 0.405 0.146 0.005 0.226 0.100 0.024 


Grade Level (Grade 9 reference) 


Grade 10 -0.055 0.188 0.772 -0.317 0.107 0.003 


Grade 11 -0.274 0.153 0.074 -0.094 0.105 0.373 


Subject Area (social studies reference) 


Mathematics 0.183 0.179 0.306 -0.026 0.149 0.860 
English language arts 0.643 0.213 0.003 -0.107 0.150 0.474 
Science 0.246 0.191 0.198 0.149 0.131 0.257 
Interdisciplinary 0.066 0.198 0.740 -0.082 0.125 0.508 
Constant 2.639 0.190 0.000 2.871 0.133 0.000 


Regression Results: Subgroup Differences in Changes in Student Agency During the School Year 


With the exception of grade level, we did not observe consistent patterns in subgroup differences 
in changes in measures of student agency during the school year. Tables 33-40 present results of 
regression analyses examining subgroup differences in changes in student agency measures 
during the school year. 


Table 33. Changes in Self-Efficacy From Fall 2017 to Spring 2018 and Levels of Self-Efficacy in Fall 
2017, Among Students Who Responded to Both Surveys 


Change From Fall to Spring (n=132) Fall Levels (n=132) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female 0.004 0.093 0.964 0.158 0.096 0.099 
Race/Ethnicity (White reference) 

Black -0.003 0.135 0.985 0.053 0.140 0.704 

Hispanic 0.196 0.134 0.144 -0.059 0.139 0.673 

Other 0.323 0.196 0.100 -0.320 0.203 0.114 


English Learner (EL) Status (non-EL reference) 


EL -0.063 0.127 0.622 -0.038 0.131 0.772 
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Change From Fall to Spring (n=132) Fall Levels (n=132) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home 0.053 0.120 0.662 0.065 0.124 0.602 


Grade Level (Grade 9 reference) 
Grade 10 0.080 0.152 0.598 0.077 0.157 0.623 
Grade 11 0.199 0.142 0.160 -0.044 0.147 0.766 


Subject Area (social studies reference) 


Mathematics -0.497 0.224 0.027 0.023 0.232 0.922 
English language arts 0.272 0.185 0.142 0.234 0.192 0.222 
Science -0.045 0.144 0.756 0.146 0.149 0.328 
Interdisciplinary 0.120 0.163 0.462 0.081 0.169 0.631 
Constant -0.082 0.155 0.597 2.923 0.160 0.000 


Table 34. Changes in Perseverance of Interest From Fall 2017 to Spring 2018 and Levels of 
Perseverance of Interest in Fall 2017, Among Students Who Responded to Both Surveys 


Change From Fall to Spring (n=131) Fall Levels (n=132) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female 0.036 0.135 0.787 0.109 0.117 0.354 
Race/Ethnicity (White reference) 

Black -0.224 0.198 0.259 0.229 0.171 0.179 

Hispanic -0.236 0.195 0.227 0.140 0.169 0.409 

Other -0.067 0.285 0.814 0.097 0.248 0.695 


English Learner (EL) Status (non-EL reference) 
EL -0.080 0.185 0.666 0.098 0.160 0.540 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home -0.113 0.175 0.518 -0.030 0.151 0.845 


Grade Level (Grade 9 reference) 
Grade 10 -0.011 0.223 0.959 -0.124 0.192 0.517 
Grade 11 -0.169 0.206 0.413 0.065 0.179 0.718 
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Change From Fall to Spring (n=131) Fall Levels (n=132) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Subject Area (social studies reference) 


Mathematics 0.139 0.326 0.671 -0.338 0.283 0.232 
English language arts -0.052 0.269 0.846 0.114 0.234 0.627 
Science -0.138 0.210 0.511 0.132 0.182 0.470 
Interdisciplinary 0.175 0.237 0.461 0.073 0.206 0.723 
Constant -0.041 0.225 0.856 2.569 0.195 0.000 


Table 35. Changes in Perseverance of Effort From Fall 2017 to Spring 2018 and Levels of 
Perseverance of Effort in Fall 2017, Among Students Who Responded to Both Surveys 


Change From Fall to Spring (n=131) Fall Levels (n=132) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female -0.227 0.115 0.048 0.157 0.113 0.164 


Race/Ethnicity (White reference) 


Black -0.128 0.169 0.448 0.167 0.165 0.310 
Hispanic -0.155 0.166 0.348 0.104 0.163 0.525 
Other -0.330 0.242 0.173 0.083 0.239 0.727 


English Learner (EL) Status (non-EL reference) 


EL 0.232 0.157 0.139 -0.301 0.155 0.052 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books athome = -0.125 0.148 0.398 0.263 0.146 0.071 


Grade Level (Grade 9 reference) 
Grade 10 -0.052 0.189 0.785 -0.098 0.185 0.596 
Grade 11 0.466 0.175 0.008 -0.170 0.173 0.325 


Subject Area (social studies reference) 


Mathematics 0.227 0.277 0.412 -0.271 0.273 0.321 
English language arts 0.140 0.229 0.541 0.308 0.226 0.173 
Science -0.024 0.178 0.892 0.210 0.176 0.233 
Interdisciplinary 0.128 0.202 0.525 0.036 0.199 0.858 
Constant 0.126 0.191 0.509 2.691 0.188 0.000 
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Table 36. Changes in Locus of Control From Fall 2017 to Spring 2018 and Levels of Locus of Control 
in Fall 2017, Among Students Who Responded to Both Surveys 


Change From Fall to Spring (n=132) Fall Levels (n=132) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female -0.107 0.081 0.186 0.085 0.090 0.348 
Race/Ethnicity (White reference) 

Black 0.068 0.118 0.566 0.059 0.132 0.655 

Hispanic -0.025 0.117 0.831 0.148 0.131 0.256 

Other 0.178 0.171 0.300 -0.186 0.191 0.329 


English Learner (EL) Status (non-EL reference) 


EL 0.168 0.111 0.131 -0.270 0.124 0.029 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books athome ~—_ -0.037 0.105 0.722 0.053 0.117 0.651 


Grade Level (Grade 9 reference) 
Grade 10 -0.084 0.133 0.525 0.065 0.148 0.663 
Grade 11 0.240 0.124 0.053 -0.172 0.138 0.212 


Subject Area (social studies reference) 


Mathematics -0.325 0.196 0.097 0.395 0.218 0.070 
English language arts 0.011 0.162 0.948 0.420 0.181 0.020 
Science -0.047 0.126 0.707 0.153 0.141 0.275 
Interdisciplinary 0.041 0.143 0.775 0.144 0.159 0.366 
Constant -0.015 0.135 0.914 2.818 0.151 0.000 


Table 37. Changes in Mastery Orientation From Fall 2017 to Spring 2018 and Levels of Mastery 
Orientation in Fall 2017, Among Students Who Responded to Both Surveys 


Change From Fall to Spring (n=132) Fall Levels (n=132) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female -0.097 0.111 0.382 0.070 0.120 0.561 
Race/Ethnicity (White reference) 

Black -0.157 0.161 0.331 0.459 0.175 0.009 

Hispanic 0.029 0.160 0.857 0.363 0.174 0.037 

Other -0.053 0.237 0.824 0.166 0.255 0.514 
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Change From Fall to Spring (n=132) Fall Levels (n=132) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


English Learner (EL) Status (non-EL reference) 


EL -0.180 0.151 0.232 -0.218 0.165 0.187 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books athome = -0.217 0.145 0.134 0.286 0.156 0.066 


Grade Level (Grade 9 reference) 
Grade 10 -0.341 0.208 0.100 0.260 0.197 0.187 
Grade 11 0.036 0.196 0.855 -0.097 0.184 0.599 


Subject Area (social studies reference) 


Mathematics -0.242 0.270 0.370 0.182 0.291 0.531 
English language arts -0.122 0.222 0.582 0.844 0.241 0.000 
Science -0.286 0.186 0.123 0.368 0.187 0.050 
Interdisciplinary -0.174 0.214 0.416 0.271 0.212 0.202 
Constant 0.483 0.212 0.023 2.057 0.201 0.000 


Table 38. Changes in Metacognitive Self-Regulation From Fall 2017 to Spring 2018 and Levels of 
Metacognitive Self-Regulation in Fall 2017, Among Students Who Responded to Both Surveys 


Change From Fall to Spring (n=128) Fall Levels (n=130) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female -0.044 0.104 0.670 0.250 0.103 0.016 
Race/Ethnicity (White reference) 

Black -0.117 0.152 0.443 0.329 0.150 0.028 

Hispanic -0.056 0.155 0.720 0.256 0.153 0.095 

Other -0.286 0.218 0.189 0.190 0.217 0.382 


English Learner (EL) Status (non-EL reference) 


EL -0.317 0.147 0.031 -0.034 0.142 0.809 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books athome ~—_— -0.176 0.137 0.198 0.300 0.136 0.028 


Grade Level (Grade 9 reference) 
Grade 10 -0.187 0.176 0.287 0.171 0.173 0.323 
Grade 11 0.267 0.159 0.094 -0.075 0.158 0.633 
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Change From Fall to Spring (n=128) Fall Levels (n=130) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Subject Area (social studies reference) 


Mathematics -0.060 0.255 0.814 -0.042 0.255 0.870 
English language arts 0.211 0.209 0.313 0.536 0.209 0.010 
Science -0.137 0.166 0.408 0.301 0.164 0.067 
Interdisciplinary 0.126 0.187 0.500 0.059 0.187 0.753 
Constant 0.302 0.177 0.088 2.062 0.176 0.000 


Table 39. Changes in Self-Regulated Learning From Fall 2017 to Spring 2018 and Levels of Self- 
Regulated Learning in Fall 2017, Among Students Who Responded to Both Surveys 


Change From Fall to Spring (n=127) Fall Levels (n=128) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female -0.083 0.107 0.436 0.220 0.119 0.066 


Race/Ethnicity (White reference) 


Black -0.024 0.153 0.876 0.357 0.172 0.038 
Hispanic -0.238 0.157 0.130 0.386 0.177 0.030 
Other -0.200 0.220 0.363 0.258 0.249 0.300 


English learner (EL) Status (non-EL reference) 


EL 0.004 0.145 0.978 -0.099 0.163 0.544 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books athome ~—_- -0.041 0.141 0.773 0.230 0.158 0.147 


Grade Level (Grade 9 reference) 
Grade 10 -0.341 0.183 0.062 0.398 0.198 0.045 
Grade 11 0.330 0.161 0.040 -0.264 0.181 0.144 


Subject Area (social studies reference) 


Mathematics -0.089 0.261 0.734 -0.136 0.292 0.641 
English language arts 0.045 0.231 0.845 0.565 0.244 0.020 
Science -0.132 0.168 0.432 0.455 0.188 0.016 
Interdisciplinary -0.106 0.195 0.586 0.407 0.217 0.060 
Constant 0.241 0.182 0.185 2.084 0.202 0.000 
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Table 40. Changes in Future Orientation From Fall 2017 to Spring 2018 and Levels of Future 
Orientation in Fall 2017, Among Students Who Responded to Both Surveys 


Change From Fall to Spring (n=126) Fall Levels (n=127) 


Standard Standard 
Coefficient Error p-value Coefficient Error p-value 


Gender (male reference) 


Female -0.028 0.117 0.809 0.236 0.132 0.074 
Race/Ethnicity (White reference) 

Black -0.004 0.167 0.981 0.325 0.190 0.087 

Hispanic -0.057 0.170 0.739 0.299 0.195 0.125 

Other -0.284 0.239 0.234 0.215 0.274 0.433 


English Learner (EL) Status (non-EL reference) 


EL -0.249 0.157 0.112 -0.078 0.179 0.665 


Socioeconomic Status (fewer than 100 books reference) 


100 or more books at home 0.104 0.152 0.495 0.197 0.174 0.256 


Grade Level (Grade 9 reference) 
Grade 10 -0.664 0.198 0.001 0.067 0.218 0.756 
Grade 11 0.222 0.174 0.202 -0.547 0.198 0.006 


Subject Area (social studies reference) 


Mathematics -0.492 0.282 0.081 0.318 0.320 0.322 
English language arts -0.609 0.258 0.018 0.695 0.274 0.011 
Science -0.362 0.181 0.046 0.537 0.206 0.009 
Interdisciplinary -0.337 0.211 0.110 0.252 0.238 0.289 
Constant 0.378 0.196 0.054 2.547 0.221 0.000 
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Data Collection Instrument: Student Survey 


Introduction 
Welcome to the Student Agency Survey! 


We want your opinion about your school, teachers, classes, schoolwork, and yourself. The only 
right answers to these questions are your honest opinions. It will take about 15 minutes to 
complete this survey. 


This survey is voluntary. If you do not want to answer a question, you may skip it, but we hope 
you will answer as many questions as you can. Your responses to survey questions will be shared 
with teachers as part of the study so that they are able to better meet the needs of students. 


Your opinions are very important to us. We appreciate your participation in this survey! 


If you agree to participate in the study, please click the “Yes” button below to continue on to the 
survey, and click the “Done” button when you are finished taking the survey. By doing so, you give 
us your permission to use your responses in our study. 


QO Yes 
© No 
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Student Information 


Student Name: 


Student ID: 


Date of Birth: 


Are you taking this survey in one of the following teachers’ classes? 


(List out the participating NIC teachers that they could select) 


Pick the teacher you saw today. 


If you didn’t have any of the teachers: 


Please type in the name of that class: 


Please select the class period: 


Please continue to think of that class/teacher as you answer the following questions in the survey. 
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To what extent do you agree or disagree with the following Strongly Strongly 


statements about you? Disagree Disagree Agree Agree 


O @ 
@ O 
@ O 
@ O 
O O 
O O 
O O 
O O 


| will be able to achieve most of the goals that | have set for myself. 


When facing difficult tasks, | am certain that | will accomplish them. 


In general, | think that | can achieve goals that are important to me. 


| believe | can succeed at most almost anything to which | set 
my mind. 


| will be able to successfully overcome challenges. 


| am confident that | can perform effectively on many different tasks. 


| can do most tasks very well. 


Even when things are tough, | can perform quite well. 


@'@@'e@ @ @,@ @ 
@|'@@)'e@ @ @'@/ @ 


To what extent do you agree or disagree with the following Strongly Strongly 


statements about you? Disagree Disagree Agree Agree 


| often set a goal but later choose to pursue a different one. O @ O © 


| have been obsessed with a certain idea or project for a short time 
but later lost interest. 


| have difficulty maintaining my focus on projects that take more 
than a few months to complete. 


New ideas and projects sometimes distract me from previous ones. 


| finish whatever | begin. 


Setbacks don’t discourage me. 


| am diligent. 


@|'@,@'0@|'e@| @ 
@|@'6@'6@,'e@ @ | @ 
@|@ @'0@'e@ © 
@|'@@'0@|'e@| @ 


| am a hard worker. 


To what extent do you agree or disagree with the following Strongly Strongly 


statements about you? Disagree Disagree Agree Agree 


Whether or not | get to be a leader depends mostly on my ability. 


When | make plans, | am almost certain to make them work. 


How many friends | have depends on how nice a person | am. 


| can pretty much determine what will happen in my life. 


| am usually able to protect my personal interests. 


When | get what | want, it’s usually because | worked hard for it. 


@©|0@ @'0@\'0'0@ @ 
@|0@ 60,06 'e@ 
@|@ @'0\'0'6 @ 
@,@ 60,06 @ 


My life is determined by my own actions. 
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To what extent do you agree or disagree with the following Strongly Strongly 


statements about you? Disagree Disagree Agree Agree 


| like classwork that I'll learn from even if | make a lot of mistakes. @ © @ © 


An important reason why | do my classwork is because | like to 
learn new things. 


| like classwork best when it really makes me think. 


get better at it. 


An important reason | do my classwork is because | enjoy it. 


O O 
O O 
An important reason why | do my classwork is because | want to O O 
O @ 
O O 


| do my classwork because |’m interested in it. 


To what extent do you agree or disagree with the following Strongly Strongly 


statements about you? Disagree Disagree Agree Agree 


During class time | often miss important points because I'm 
thinking of other things. O O O O 


When | become confused about something I'm reading for this 
class, | go back and try to figure it out. 


If class materials are difficult to understand, | change the way | 
read the material. 


Before | study new class material thoroughly, | often skim it to see 
how it is organized. 


| ask myself questions to make sure | understand the material | 
have been studying in this class. 


| try to change the way | study in order to fit the class requirements 
and instructor's teaching style. 


| often find that | have been reading for class but don't know what 
it was all about. 


| try to think through a topic and decide what | am supposed to 
learn from it rather than just reading it over when studying. 


When studying for this class | try to determine which concepts | 
don't understand well. 


When | study for this class, | set goals for myself in order to direct 
my activities. 


If | get confused taking notes in class, | make sure | figure it out 
afterwards. 


@® @- @2' @ @'@,'|8@/|@/| @| @ 
@ @~ @2' 082 @'@'@/'@,| @| ®@ 
@@e- @2 @ @,'@'@/'|@| @| @ 
@®@-@2 @2 @'8@'8@|'@| @| @ 
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To what extent do you agree or disagree with the following Strongly Strongly 


statements about you? Disagree Disagree Agree Agree 


O O 
O O 
O O 
@ @ 
O O 
O O 
O @ 
O O 
@ O 


| set aside time to do my homework and study. 


| try to do well on my schoolwork even when it isn’t interesting to me. 


If | need to study, | don’t go out with my friends. 


| always study for tests. 


| keep track of my long-term assignments so | know when to turn 
them in. 


| manage my time well enough to get all my work done. 


| can keep my schoolwork and personal life organized. 


| set goals for my performance in classes. 


@'@,@'e@| @ 006 @ 
@|0@'@'e@| @ 0|'0@'0 @ 


| have a system for organizing my schoolwork. 


To what extent do you agree or disagree with the following Strongly Strongly 


statements about you? Disagree Disagree Agree Agree 


Grades in high school matter for success in college. 


My classes give me useful preparation for what | plan to do in life. 


High school teaches me valuable skills. 


Working hard in high school matters for success in the workforce. 


@|@ @|e@|e@ 
@|@ 6'e@|@ 
@|@ 6|0|@ 
@|@ @|0@\e@ 


What | learn in class is necessary for success in the future. 
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Please think of the teachers in your school as you answer the following questions in the survey. 


To what extent do you agree or disagree with the following Strongly Strongly 


statements about the teachers in your school? Disagree Disagree Agree Agree 


My teachers really listen to what | have to say. 


My teachers will discuss my grades with me. 


My teachers notice when | am having trouble learning something. 


My teachers check to make sure we understand what s/he is 
teaching us. 


My teachers will help me catch up if | am behind. 


My teachers explain difficult things clearly. 


My teachers believe | can do well in school. 


My teachers pay attention to all students, not just the top students. 


My teachers have several good ways to explain things. 


My teachers will give me extra help on schoolwork if | need it. 


@|'@ @'0@'0 60 © 66 6 
@,@@ 0 @6¢ 6 @ @ 66 
@@@@e@6@@ @ @ 66 
@,e@@@,e@@ 0 @ 66 @ 


My teachers will help me stay busy and interested if | get ahead. 


Student Demographics 
Are you a male or female? 
) Male 
_) Female 
Are you Hispanic or Latino? 
) Yes, | am Hispanic or Latino. 
| > No, | am not Hispanic or Latino. 
Which of the following best describes you? 
_) White 
Black or African American 
Asian 
American Indian or Alaska Native 
Native Hawaiian or other Pacific Islander 


Multiracial 


VU YY 
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How often do you speak English at home? 
QO Always (If Always, skip next question). 
QO Almost always 
QO Sometimes 


QO Never 


What language other than English do you speak at home? 


QO Spanish 


QO Other (Please specify: ) 


About how many books are there in your home? (Do not count magazines, newspapers, 
or your schoolbooks.) 


QO None or very few (O-10 books) 

QO Enough to fill one shelf (11-25 books) 

QO Enough to fill one bookcase (26-100 books) 

QO Enough to fill two bookcases (101-200 books) 

QO Enough to fill three or more bookcases (more than 200) 


How many digital information devices are there in your home? Count computers, tablets, 
smartphones, smart TVs, and e-readers. (Do not count other devices.) 


QO None 
QO 1-3 devices 


CO) 4-6 devices 
©) 7-40 devices 


QO More than 10 devices 
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Data Collection Instrument: Teacher Survey 


Introduction 


New Tech Network is working with the American Institutes for Research (AIR) to study teacher 
practices that support the development of student agency. The study aims to identify the 
instructional practices that are particularly useful for the development of different aspects of 
student agency (i.e., self-efficacy, self-regulated learning, and persistence) and whether these 
instructional practices are equally helpful for different subgroups of students. As part of this study, 
AIR is surveying teachers in four high schools in the New Tech Network. 


The survey questions ask for your opinions and experiences related to your school, students, 
and instruction. By completing the survey, you agree to allow your responses to be included 
in the study. 


This survey will not be used to evaluate you or anyone else in your school. 


The survey is voluntary and confidential. We will not and cannot share individual responses with 

anyone outside of the study team at AIR. Although your data will be linked to your email address, 
no one outside of AIR will have access to that information. Findings will be reported in groups of 
responses among the schools participating in the study as well as at the school level. 


Responses will not be used to evaluate your school or to compare schools, but will be used to 
understand behaviors and perspectives of the teachers in this study. 


If you do not want to answer a question, you may skip it, but we hope you will answer as many 
questions as you can. Your perspective is very important to us. 


The survey will take approximately 15 minutes to complete. 


If you agree to participate in the study, please click the “Yes” button below to continue on to the 
survey, and click the “Done” button when you are finished. By doing so, you give us your permission 
to use your responses in our study. 


QO Yes 
QO No 
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NIC Participation 


Are you participating in the networked improvement community (NIC) related to this study? 


7, Yes 
-, No 


Instructions 


The following questions ask about your general perception of other teachers at your school in the 
areas of innovation, improvement, and commitment. When you answer, think about your experience 
working at your current school. 


If this is the first year you have worked at your school, please check the box below. 
L} Yes, this is my first year teaching at this school. 


Please mark the extent to which you disagree or agree with each Strongly Strongly 


of the following: Disagree Disagree Agree Agree 


1. | usually look forward to each working day at this school. @ @ @ @ 
2. | wouldn't want to work in any other school. O @ © © 
3. | feel loyal to this school. © O © O 
4. | would recommend this school to parents seeking a place for 

their child. O @ O @ 
5. | would want to have my child in this school. O @ @ O 


For this set of questions, think about the teachers in your school. 
Strongly Strongly 


To what extent do you disagree or agree with the following? Disagree Disagree Agree Agree 


6. Once we start a new program in this school, we follow up to 
make sure that it’s working. O ® O O 


7. We have so many different programs in this school that | can’t 
keep track of them all. 


8. Many special programs come and go at this school. 


9. Curriculum, instruction, and learning materials are well 
coordinated across the different grade levels at this school. 


10. There is consistency in curriculum, instruction, and learning 
materials among teachers in the same grade level at this school. 


@  @ |e @ 
@ | @ |e @ 
@ | @ | @ @ 
@ | @ |e @ 
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For this set of questions, think about the teachers in your school. 


About 
How many teachers at this school: None Some Half Most Nearly All 
11. Are really trying to improve their teaching? O @ @ O © 
12. Are willing to take risks to make the school 
better? O O O ® O 
13. Are eager to try new ideas? O () © O O 


For this set of questions, think about the teachers in your school. 


Please mark the extent to which you disagree or agree with each 


of the following. Strongly Strongly 
Teachers in my school: Disagree Disagree Agree Agree 


14. Have made changes designed to better meet the needs of the 


school’s diverse student body. O O O O 
15. Are engaged in systematic analysis of teaching practices. @ © @ O 
16. Have well-defined plans for instructional improvement. O @ O @ 
17. Openly examine and acknowledge progress towards an 

instructional vision. e) O O O 
18. Are engaged in systematic analysis of student-performance 

data. O O O O 
19. Review student learning and understanding in order to adjust 

their practices. O O O O 
20. Have a clear vision of instruction linked to standards for 

student learning and growth. O O O ® 

Think about the typical students you teach in this school. 
About Nearly 

How many students: None Some Half Most All 


21. Come to class on time? 


22. Attend class regularly? 


23. Come to class prepared with the appropriate 
supplies and books? 


24. Regularly pay attention in class? 


25. Actively participate in class activities? 


@;ee@ @ @'@ 
@\ee@ @ @|'@ 
@;e@'@ @ 6'@ 
@ee@ @ 6 '@ 
@\@6@ @ 6'@ 


26. Always turn in their homework? 
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Nothing or A Fair A Great 


How much can you do to: Very Little Some Amount Deal 


27. Overcome the influence of adverse community 
conditions on students’ learning? O O O O 


28. Promote learning when there is a lack of support from 
the home? 


29. Control disruptive behavior in the classroom? 


30. Motivate students who show low interest in schoolwork? 


31. Get through to the most difficult students? 


32. Get students to work together? 


33. Keep students on task on difficult assignments? 


@|0@ @'0@)'e/'e@| @ 
@|0@ 6 ee @| @ 


34. Get students to do their homework? 


A few 
A few times 


How often is each statement below times per Every 
true about you? Never peryear Monthly month VeX=) ¢ 


35. | can easily understand how my O O @ O @ @ @ 


students feel about things. 


36. | deal very effectively with the 
problems of my students. 


37. | feel I’m positively influencing my 
students’ lives through my work. 


38. | feel very energetic. 


39. | can easily create a relaxed 
atmosphere with my students. 


AO. | feel exhilarated after working 
closely with my students. 


41. | have accomplished many 
worthwhile things in this job. 


42. In my work, | deal with emotional 
problems very calmly. 


@® @|@| @/@ @| @ 
@® @| @| @ |e @| @ 
® @|@| @ | @| @| @ 
@® @| @| @ 6 @ | ®@ 
@® @,'|@| @/@| @| @ 
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A few 
A few times 


How often is each statement below times per Every 


month week 


A few 
times 
per Every 


true about you? Never peryear Monthly 


43. | feel | treat most students as 
@ O @ O O 


respected individuals. 


week day 


44. |’'ve become more callous 
towards people since | took this © CO 
job. 


hardening me emotionally. 


O 
45. | worry that this job was O O O 
@ 


46. | really care what happens to 
most students. O O 


O 


47. | feel students blamed me for 
O O O @ O 


some of their problems. 


@® @,'@e@ | @ 


O 


NOTE: Some questions include the term “agency.’ By agency, we mean “students’ capabilities to manage their own learning and be 


successful in school.” 


For a typical class, how often do you use the following practices 1-3 


times a 
month 


with most of your students? 


Provide students with opportunities to... Rarely 


More 


1-3 than 3 
times a 
week week 


times a 


48. Make connections between outside agency and its application 

in the classroom. O O O O 
49. Revise assignments or tests after they have received feedback. @ @ @ @ 
50. Self-reflect using journals, logs or other structured templates 

or tools. O O O O 
51. Lead instruction on a particular skill or concept. @ O @ @ 
52. Contribute to and provide feedback on key decisions in the 

Classroom. O O O O 


NOTE: Some questions include the term “agency.” By agency, we mean “students’ capabilities to manage their own learning and be 


successful in school.” 


1-3 
times a 
month 


For a typical class, how often do you use the following practices 
with most of your students? 


Rarely 


More 


1-3 than 3 
times a 
week week 


times a 


53. Develop personal relationships with students to better 

understand their agency strengths, needs, and motivators. O O O O 
54. Guide students in the process of asking for feedback. O O O © 
55. Help students set goals to complete coursework while O O O @ 


improving their agency to do so on their own. 
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More 
1-3 1-3 than 3 


For a typical class, how often do you use the following practices timesa timesa_ timesa 
with most of your students? Rarely month week week 


56. Hold one-on-one meetings with students to discuss elements O O O @ 
of student agency and its relationship to academic work. 


57. Design formative and summative assessments to evaluate 
student agency. 


58. Provide students with extrinsic motivation to build agency 
skills. 


59. Provide explicit instruction to develop skills related to student 
agency. 


60. Model agency skills to demonstrate those skills to students in 
a meaningful context. 


61. Provide positive reinforcement for demonstration of agency 
skills. 


62. Provide students with tools, strategies, and resources to coach 
them towards mastery of agency skills. 


® @,'@;@ @| @| ®@ 


63. Provide brief spoken prompts in real time to highlight or O O O 
remind students of behaviors that demonstrate agency. 


NOTE: Some questions include the term “agency.’ By agency, we mean “students’ capabilities to manage their own learning and be 
successful in school.” 


How many students in your school have the following About 
opportunities? None Some Half 
64. Make choices about the content and process of 

their work. O @ O @ 


65. Work in groups to learn and practice agency skills 
necessary for group success. 


66. Demonstrate agency outside the classroom (in 
the school or in the community). 


67. Make connections between outside agency and 
its application in the classroom. 


68. Revise assignments or tests after they have 
received feedback. 


69. Self-reflect using journals, logs, or other 
structured templates or tools. 


70. Demonstrate agency by leading instruction on a 
particular skill or concept. 


@® @'@'@,| @| ®@ 
@® @,' @'@;|@| ®@ 
@® @,'| @' @/| @| @ 
@® @| | @' @;| @ | @| @ 
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Teacher Demographics 


1. What grade(s) do you teach? (Select all that apply.) 


© oth 
©) 1oth 
©) 11th 
©) 12th 


2. What subject(s) do you teach? (Select all that apply.) 


QO English language arts 
QO Mathematics 


QO Science 


QO Social studies 


©) History 


QO Foreign language 


QO Other (please specify): 


3. How many students do you teach in a typical day? 


QO Fewer than 50 


OC) 50 to 74 
©) 75 to 100 


QO More than 100 


4. What is the highest degree you have earned? 
©) High school diploma 
QO Associate’s degree 
QO Bachelor’s degree 
O Master’s degree 
QO Educational specialist diploma 
QO Ed.D., Ph.D., law degree, or other high-level professional degree 


QO | do not have a college degree. 
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Including this school year (2017-18), how many 11 years 


years have you taught (count this school year as 1): lyear 2-3 years 4-5 years 6-10 years  ormore 


Any grade from K-8 at any school? @ @ © @ @ 
Any grade from 9-12 at any school? @ @ @ @ O 
Any subject at any grade level at this school? @ O @ O @ 
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